Papers with BIRD benchmark
MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL (2025.coling-main)
Copied to clipboard
Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, Zhoujun Li
| Challenge: | Recent LLM-based Text-to-SQL methods suffer from performance degradation on “huge” databases and complex user questions that require multi-step reasoning. |
| Approach: | They propose a framework that integrates a decomposer agent and auxiliary agents to generate SQL queries from natural language text. |
| Outcome: | The proposed framework achieves comparable execution accuracy on SQL-Llama tasks compared to the baseline model. |
SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches depend on static, pre-processed database information, which restricts the model’s capacity to deeply comprehend the underlying database content. |
| Approach: | They propose a framework that empowers LLMs to perform Self-Driven Exploration of databases during inference. |
| Outcome: | Evaluated on the BIRD benchmark with Qwen2.5-72B-Instruct, SDE-SQL achieves an 8.02 % improvement in execution accuracy over the baseline. |
Divide, Link, and Conquer: Recall-oriented Schema Linking for NL-to-SQL via Question Decomposition (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Open-source LLMs often depend on large proprietary models, which introduce serious privacy concerns. |
| Approach: | They propose a plug-and-play framework that improves SQL generation for smaller LLMs . they propose to apply question decomposition at the schema linking stage rather than during SQL generation . |
| Outcome: | The proposed framework improves schema linking recall by 25.1% and execution accuracy by 8.2% on the BIRD benchmark. |
GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL (2026.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for Text2SQL generation still have a critical semantic gap . a dedicated validator translates generated SQL back into natural language and checks whether its logic is aligned with the original question. |
| Approach: | They propose a framework that introduces Guided Generation with SQL2Text Back-translation Validation . dedicated validator translates generated SQL back into natural language and checks whether logic is aligned with original question . |
| Outcome: | The proposed framework achieves 63.23% execution accuracy on the BIRD benchmark and 90.42% on repaired BIDR dev. |
SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved strong performance on natural language to SQL (NL2SQL) benchmarks, yet their reported accuracy may be inflated by contamination from benchmark queries or structurally similar patterns seen during training. |
| Approach: | They propose a syntactic probing framework for detecting and quantifying such contamination in large language models. |
| Outcome: | The proposed framework generates syntactic variants of test queries for four widely used NL2SQL datasets. |